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INDEPENDENT  RESEARCH  QR)  PROGRAMS 


The  Navy  Personnel  Research  and  Development  Center  (NAVPERSRANDCEN)  has 
been  conducting  Independent  Research  (IR)  since  its  establishment  in  1973.  The  resources 
provided  for  this  program  have  been  used  to  develop  research  methods  and  techniques 
related  to  training,  employment,  and  performance  of  people  in  the  Navy.  In  FYs  81  and 
82,  IR  work  has  been  concentrated  in  the  areas  indicated  in  Table  1.  Selected  projects 
within  these  areas  are  described  in  the  remainder  of  this  section. 

IR/IED  work  units  for  FY83  are  presented  in  Appendix  A;  and  IR/IED  publications  and 
presentations,  in  Appendix  B. 


Table  1 

IR  Work  Units  for  FY81-82 


Dollars  (in 

Independent  thousands) 


Research 

FY81 

FY82 

ZROQO-0 1-042-04: 

SELECTION  AND  RETENTION 

-01 

Measurement  Theory  and  Methods 
in  Personnel  Research 

83 

68 

-01.03 

Differential  Worth  for 

Optimal  Individual 

Placement3 

Dr.  L.  Kroeker 
Code  12 

10 

5 

-01.04 

Unobtrusive  Measures  of 

Attitude 

Dr.  C.  A.  Robinson 
Code  14 

12 

0 

-01.05 

Methods  for  Clustering 

Tasks 

Mr.  R.  Chatfield 
Code  12 

15 

0 

-01.06 

Models  for  Personnel 

Management 

Mr.  3.  Silverman 
Code  11 

40 

30 

-01.07 

Models  for  Calibrating 
Multiple-choice  Items 

Dr.  3.  B.  Sympson 
Code  12 

6 

0 

-01.08 

Cognitive  Speed  Tests 

Dr.  B.  Rimland 

Mr.  G.  E.  Larson 
Code  01C 

0 

21b 

-01.09 

Adaptive  Ability  Tests 

Dr.  D.  Wetzel 

Code  12 

0 

3 

-01.10 

Models  and  Measures  of 

Human  Performance 

Dr.  R.  C.  Sorenson 
Code  03 

0 

9C 

aDescribed  in  detail  in  previous  IR/IED  report,  NPRDC  SR  82-27  (Rimland,  1982),  (AD- 
A 117  630). 

bBeing  continued  under  FY83  work  unit  ZROOG-0 1-042.003  (see  Appendix  A). 
cBeing  continued  under  FY83  work  unit  ZR000-01-042.005  (see  Appendix  A). 
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Table  1  (Continued) 


Independent 

Research 

Dollars  (in 
thousands) 
FY81  FY82 

-03 

Retention  of  Quality  Personnel 

95 

0 

-03.01 

Methods  for  Projecting  Petty 
Officer  Retention 

Dr.  E.  Curtis 

Code  16 

65 

0 

-03.02 

The  Reenlistment  Decision 

Process 

Dr.  L.  M.  Doherty 
Code  301 

30 

0 

-042-06: 

LEARNING  AND  TRAINING 

-01 

Instructional  Psychology 

77 

141 

-01.01 

Enhancement  of  Information 
Acquisition  and  Storage® 

Dr.  3.  Ellis 

Code  13 

20 

42 

-01.02 

Delayed  Feedback  in 

Acquisition  and  Retention 

Dr.  N.  Van  Matre 
Code  14 

13 

0 

-01.03 

Improving  Language-learning 
and  Problem-solving  Abilities3 

Dr.  R.  Wisher 

Code  13 

44 

5 

-01.04 

Metaphors  and  Analogies  in 
Training 

Dr.  S.  Hearold 
Code  15 

0 

30 

-01.05 

Cognitive  Storage  Mechanisms 
and  Learning  to  Read 

Dr.  M.  Baker 

Code  15 

0 

50 

-01.06 

Oculomotor  Training  for  Pilots 

Mr.  W.  Thode 

Code  14 

0 

I4b 

-042-08: 

GROUP  BEHAVIOR 

-03 

Group  Performance  Effectiveness 

0 

51 

-03.01 

Group  Problem-solving  in 
Operational  Settings 

Dr.  W.  Montague 

Dr.  E.  Hutchins,  Jr. 

Code  13  0 

1 

-03.02 

Organizational  Effectiveness 
of  Program  Management  Offices 

Mr.  T.  Enderwick 
Mr.  T.  Koslowski 
Code  16 

0 

50 

TOTAL 

255 

260 

described  in  detail  in  previous  IR/1ED  report,  NPRDC  SR  82-27. 
bBeing  continued  under  FY83  work  unit  ZR0OO-01-042.010  (see  Appendix  A). 
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Measures  Of  Task  Proximity 


Background 

Cluster  analysis  is  frequently  used  to  group  together  occupations  or  jobs  that  have 
similar  tasks,  attributes,  or  other  work-related  requirements.  Such  cluster  profiles  are 
useful  for  aligning  training  curricula  with  actual  work  performed  and  for  streamlining 
occupational  classification  systems  by  combining  administratively  separate  jobs  into  one 
actual  job  type.  The  Comprehensive  Occupational  Data  Analysis  Program  (CODAP)  was 
developed  by  the  Air  Force  to  fill  these  needs  and  is  widely  used  by  both  military  and 
civilian  organizations.  The  data  sets  analyzed  are  typically  responses  of  job  analysts  or 
job  incumbents  to  items  within  a  structured  task  inventory.  An  important  decision  that 
can  affect  the  cluster  analysis  solution  is  the  selection  of  a  proximity  measure  to  assess 
the  degree  of  similarity  among  patterns  of  responses. 

All  U.S.  military  se;  vices  use  the  powerful,  computer-based  clustering  procedures 
available  <n  CODAP.  While  recent  research  has  suggested  that  simple  binary  measures 
may  be  able  to  capture  as  much  profile  information  as  do  continuous  measures,  cluster 
solutions  have  not  been  compared  empirically  by  applying  various  proximity  measures  to 
occupational  data.  This  IR  project  was  directed  toward  making  such  a  comparison. 

Approach 

Data  collected  by  the  Navy  Occupational  Development  and  Analysis  Center  consisted 
of  three  samples  of  job  incumbent  profiles.  Each  sample  consisted  of  250  profiles, 
indicating  time  spent  on  various  tasks  and  based  on  a  5-point  scale.  For  each  sample,  16 
proximity  matrices  were  derived  with  each  matrix  based  on  the  application  of  one  of  3 
continuous  or  13  binary  proximity  measures.  The  CODAP  clustering  procedure  was 
applied  to  seven  selected  proximity  matrices  from  each  of  the  three  samples.  Evaluation 
of  the  binary  measures  was  based  on  the  extent  to  which  the  binary  matrices  and  cluster 
solutions  were  objectively  similar  to  those  based  on  continuous  measures. 

Conclusions 


1.  The  Jaccard  and  Dice  (see  Pass  <5c  Chatfield,  1982)  proximity  measures  are 
consistently  powerful,  capable  of  capturing  more  profile  information  than  many  other 
binary  measures. 

2.  The  performance  of  a  proximity  measure  in  cluster  analysis  can  be  strongly 
affected  by  the  proportion  of  zeros  in  the  data  analyzed. 

3.  The  use  of  selected  binary  proximity  measures  yields  cluster  solutions  highly 
similar  to  those  based  on  continuous  proximity  measures. 

4.  Since  the  high  proportion  of  zeros  in  the  incumbent  profiles  analyzed  are  typical 

for  this  type  of  data  set,  the  findings  are  generalizable  to  data  collected  from  other 
occupational  inventories.  j 

j 

Application  \ 

Based  upon  this  methodological  study,  the  Jaccard  proximity  measure  was  adopted 
and  programmed  into  CODAP  System  80,  an  enhanced  version  of  CODAP  developed  by  the 
Department  of  Defense  that  is  currently  operational  in  the  Navy  and  Marine  Corps.  j 


Models  for  Personnel  Management 


Purpose 

The  management  of  human  resources  as  a  policy  and  planning  enterprise  is  a 
relatively  new  phenomenon.  The  purpose  of  this  effort  was  to  assess  the  state  of  the  art 
in  human  resource  management  for  both  public  and  private  personnel  systems. 

Approach 

The  small  body  of  pertinent  scientific  literature  was  reviewed  and  a  managerial 
perspective  was  derived  from  the  theoretical  and  technical  features  of  manpower  planning 
research  and  practice. 

Accomplishments 

A  case  study  was  developed  that  centers  on  the  problems  of  acquiring  and  retaining 
engineering  manpower.  Problems  occur  when  a  high  technology  firm  is  acquired  by  a 
larger,  more  conventional  manufacturing  concern.  This  case  study  illustrates  many  of  the 
interrelated  factors  involved  in  managing  the  internal  labor  market  of  an  organization. 

Along  with  the  case  study,  the  internal  personnel  structure  and  flow  properties  of 
public  and  private  manpower  systems  were  examined.  After  a  description  of  internal 
supply  and  demand,  consideration  was  given  to  (1)  the  rationale  for  personnel  planning,  (2) 
the  relationship  of  a  personnel  system  to  its  external  environment,  (3)  the  state  of  the 
personnel  inventory,  (4)  methods  for  forecasting  personnel  losses  and  determining  gains, 
and  (5)  methods  for  controlling  the  personnel  system. 

The  case  study  and  the  assessment  of  personnel  systems  in  terms  of  internal  supply 
and  demand  represent  four  chapters  of  a  book  being  prepared  on  manpower  planning. 


Note.  This  work  was  accomplished  under  FY81-82  work  unit  ZROOO-0 1-042-04-0 1 .06: 
Models  for  Personnel  Management. 
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A  New  Model  for  Calibrating  Multiple-choice  Items 


Background 

The  effectiveness  of  personnel  selection  procedures  based  on  objective  assessments 
of  individual  aptitudes  and  personality  characteristics  has  been  well  documented.  Apti¬ 
tude  measurement,  in  particular,  has  been  widely  used  in  industry,  the  civil  services, 
and  the  military.  The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB),  for 
example,  is  currently  the  principal  mechanism  for  identifying  candidates  for  military 
enlistment  who  are  likely  to  succeed  in  various  military  occupations. 

Since  about  1920,  the  practice  of  personnel  testing  has  depended  heavily  on  advances 
in  the  theory  of  test  reliability  and  validity.  These  advances  have  resulted  in  various 
improvements  in  the  traditional  number-correct-scored  personnel  test,  but,  in  recent 
years,  it  has  become  increasingly  apparent  that  further  improvements  in  the  assessment 
of  individual  aptitudes  require  new  theoretical  developments. 

Item  Response  Theory 

In  the  last  10  years,  a  new  theoretical  approach  to  test  construction,  scoring,  and 
evaluation  has  emerged  that  may  lead  to  important  new  advances  in  personnel  testing. 
This  approach  is  known  as  item  response  theory  (IRT).  The  basic  assumption  underlying 
IRT  is  that  the  probability  of  selecting  a  particular  response  to  a  test  question  is  related 
to  a  person's  level  of  ability  (or  other  relevant  personal  characteristic)  by  a  specified 
mathematical  function.  What  differentiates  various  implementations  of  IRT  is  the  form 
of  the  mathematical  function  that  is  assumed. 

The  simplest  IRT  models  assume  that  the  probability  of  a  correct  response  to  a  test 
question  is  a  function  of  only  two  parameters— a  person  parameter  that  indexes  ability  and 
an  item  parameter  that  indexes  item  difficulty.  A  more  adequate  IRT  model  that  is 
widely  used  incorporates  these  parameters  plus  two  more  item  parameters.  One  of  the 
additional  item  parameters  takes  into  account  the  fact  that  low  ability  people  can 
sometimes  answer  a  multiple-choice  question  correctly  by  guessing.  The  other  added  item 
parameter  provides  an  index  of  the  discriminating  power  of  the  item.  In  applications  of 
this  model,  items  that  are  known  to  discriminate  well  between  low  and  high  ability  people 
are  given  more  weight  when  an  individual's  level  of  ability  is  estimated. 

The  IRT  models  described  above  have  two  drawbacks.  First,  they  classify  responses 
to  aptitude  questions  as  either  correct  or  incorrect.  They  make  no  distinction  among  the 
different  incorrect  responses  that  a  person  might  select.  Information  about  a  person's 
level  of  ability  that  could  be  extracted  by  taking  into  account  which  particular  incorrect 
responses  have  been  selected  is  lost  when  these  models  are  used.  Second,  these  models 
assume  that  the  probability  of  a  correct  response  to  an  item  is  a  strictly  increasing 
monotonic  function  of  ability;  that  is,  the  probability  of  a  correct  response  always 
increases  as  ability  increases.  In  our  research,  evidence  suggests  that  this  assumption  is 
too  restrictive. 

Approach 

Data  were  analyzed  from  125  word  knowledge  items  and  145  arithmetic  reasoning 
items  that  had  appeared  in  operational  forms  of  the  Scholastic  Aptitude  Test  (SAT), 
published  by  the  Educational  Testing  Service.  For  most  of  the  items,  the  proportion  of 


Note.  This  work  was  accomplished  under  FY81  work  unit  ZROOO-01-042-04-01.07;  Models 
for  Calibrating  Multiple-choice  Items.  , 


correct  responses  tended  to  increase  monotonically  as  ability  increased.  However,  for 
about  one  item  in  six,  the  proportion  of  correct  responses  tended  to  decline  as  ability 
increased  from  very  low  to  moderate  levels,  and  then  to  increase  as  ability  rose  from 
moderate  to  high  levels.  This  finding  was  attributed  to  the  presence  in  some  of  the  items 
of  one  or  more  "plausible"  incorrect  responses  that  were  more  attractive  to  examinees  at 
middle  ability  levels  than  to  those  at  low  ability  levels. 

If  a  significant  number  of  items  in  a  given  aptitude  domain  have  nonmonotone 
regressions  of  the  type  described  above,  it  is  clear  that  current  1RT  models  cannot  provide 
an  adequate  basis  for  personnel  testing  in  that  domain.  In  order  to  deal  with  this  problem, 
a  new  IRT  model  based  on  the  multivariate  logistic  function  was  developed. 

The  new  IRT  model  takes  into  account  the  characteristics  of  each  item  response. 
The  various  incorrect  responses  to  a  multiple-choice  question  are  kept  distinct,  and  the 
tendency  for  people  at  different  ability  levels  to  find  different  responses  differentially 
attractive  is  reflected  in  the  parameters  of  the  model.  Also,  the  model  is  flexible  enough 
to  allow  fitting  of  nonmonotone  item-ability  regressions  when  they  occur.  The  new  model 
has  been  applied  to  selected  SAT  items  and  found  to  fit  the  available  data  very  well. 

Conclusions 

If  this  new  IRT  model  is  adopted  for  the  calibration  of  test  items,  two  immediate 
benefits  can  be  realized.  First,  fewer  items  will  have  to  be  rejected  for  lack  of  fit  to  the 
model.  More  items  can  be  retained  for  the  construction  of  new  aptitude  tests.  Second, 
once  tests  have  been  assembled,  the  use  of  information  about  which  incorrect  alternatives 
an  individual  has  selected  will  improve  our  estimates  of  ability.  This  will  increase  the 
reliability  of  tests  of  any  given  length  and  will  also  allow  the  use  of  shorter  tests  to 
achieve  prespecified  levels  of  reliability.  The  latter  possibility  could  lead  to  significant 
savings  in  testing  time. 


Cognitive  Speed  and  Learning  Potential 


Background 

There  is  a  long-standing,  widely  recognized  need  for  a  method  of  testing  intelligence 
that  is  fast,  easy  to  administer,  culture-free,  and  predictive  of  meaningful  criteria,  such 
as  completion  of  or  achievement  in  training. 

Current  military  testing  programs  employ  conventional  paper-and-pencil  tests  that 
are  basically  similar  to  those  used  in  World  War  II.  While  useful,  such  tests  have  long 
since  reached  the  upper  limit  of  their  utility.  Recent  developments  in  the  fields  of 
cognitive  science  and  computerized  adaptive  testing  offer  considerable  promise  for 
improving  the  technology  for  discovering  and  measuring  an  individual';  talents  and 
aptitudes.  In  particular,  cognitive  scientists  are  finding  that  simple  tests  that  provide  an 
index  of  how  quickly  and  accurately  a  person  can  perceive  a  simple  stimulus  may  yield  a 
good  deal  of  information  about  the  person's  underlying  mental  capacity. 

Approach 

The  purpose  of  this  work  is  to  develop  computer  tests  of  cognitive  speed  and  to  assess 
their  effectiveness  as  predictors  of  performance.  Two  hypothesized  measures  of 
cognitive  speed,  inspection  time  (IT)  and  reaction  time  (RT),  are  of  particular  interest. 

IT  tests  generally  require  the  subjects  to  discriminate  between  rapidly  presented 
stimuli,  usually  lines.  A  typical  display  might  be  two  horizontal  lines  of  unequal  length, 
with  the  subject  simply  being  asked  which  line  was  longer.  The  exposure  duration  for  the 
lines  is  gradually  reduced  until  the  subject's  threshold  is  found,  with  threshold  defined  as 
the  duration  in  milliseconds  at  which  the  subject's  judgment  is  correct  on  a  predetermine^ 
percentage  of  trials  (usually  97.5%;  Nettlebeck,  1982).  An  analogous  series  of  trials  using 
vertical  lines  is  sometimes  employed  along  with,  or  in  place  of,  horizontal  lines.  Forward 
and  backward  masking  ensures  that  responses  are  based  on  information  that  has  actually 
been  encoded  rather  than  on  afterimages  or  other  peripheral  (as  opposed  to  central) 
processes.  The  mask  is  a  longer  line  that  the  subject  perceives  as  covering  the  two 
stimulus  lines  before  and  after  their  presentation.  The  subject's  resultant  inspection  time 
or  threshold  value  is  thereby  presumed  to  be  largely  the  result  of  speed  of  information 
processing. 

Experimental  studies  using  inspection  time  procedures  have  found  significant  corre¬ 
lations,  ranging  from  -.25  to  -.95,  between  IT  and  more  traditional  measures  of 
intelligence.  Further,  IT  remains  fairly  stable  from  one  experiment  to  another  and  does 
not  appear  to  be  influenced  by  practice.  A  recent  review  by  Nettlebeck  (1982),  however, 
suggests  that  the  reported  correlations  between  IT  and  IQ  measures  may  have  been 
inflated  by  including  subjects  with  lower  than  average  intelligence.  It  is  possible  that  the 
high  correlations  reported  were  the  consequence  of  an  excessively  wide  spread  of  IQ 
scores,  which  dipped  into  and  below  the  mildly  retarded  range.  Nettlebeck  (1973,  1982) 
reports  a  correlation  of  -.13  between  IQ  and  IT  in  nonretarded  subjects  of  above  average 
intelligence  (IQ  from  112  to  135+),  and  -.34  for  a  group  of  college  students  of  average  and 


Note.  This  work  was  accomplished  under  FY82  work  unit  ZR000-01-042-04-01.08:  Cogni¬ 
tive  Speed  Tests. 
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above  average  intelligence.  The  present  study  will  attempt  to  determine  if  IT  measures 
provide  useful  data,  given  the  range  of  intelligence  levels  found  in  military  populations. 
Further,  by  analyzing  IT  measures  in  conjunction  with  scores  from  traditional  tests,  it  is 
hoped  to  gain  a  better  understanding  of  cognitive  functioning. 

RT  is  another  measure  of  cognitive  speed.  Several  investigators  have  reported 
promising  results  using  RT  measures  (e.g.,  Carlson  6c  Jensen,  1982;  Jensen,  1980).  The 
most  basic  RT  experiment  involves  measuring  the  linear  increase  in  reaction  time  to 
visual  or  auditory  stimuli  as  a  function  of  the  amount  of  information  presented.  A  typical 
console  RT  display  is  shown  in  Figure  1. 


Figure  1.  A  typical  RT  display  console. 


The  subject  presses  the  home  button  until  a  light  associated  with  any  one  of  the  eight 
stimulus  buttons  is  activated.  When  the  light  comes  on,  the  subject  responds  by  removing 
his  finger  from  the  home  button  and  pressing  the  associated  target  button  as  quickly  as 
possible.  Reaction  time  in  this  case  refers  to  the  interval  between  onset  of  the  light  and 
removal  of  the  subject's  finger  from  the  home  button. 

Progress 

With  the  assistance  of  Professors  Tom  Boyle  of  Purdue  University  and  Christopher 
Brand  of  the  University  of  Edinburgh,  who  were  visiting  faculty  members  at  NAVPERS- 
RANDCEN  during  the  summer  of  1982,  a  battery  of  10  computer-presented  tests  of 
inspection  and  reaction  time  were  reprogrammed  on  a  TRS-80  microcomputer.  The  test 
battery  is  being  administered  to  a  sample  of  150  incoming  students  at  the  Basic 
Electricity  and  Electronics  School  at  the  Naval  Training  Center,  San  Diego.  When  the 
testing  is  complete,  the  scores  will  be  analyzed  in  conjunction  with  school  grades  and 
ASVAB  scores  to  evaluate  the  cognitive  speed  tests  as  predictors  of  school  performance. 

The  IT  tests  are  adaptive;  that  is,  the  computer  presents  the  subject  with  an  easier 
item  if  he  responds  incorrectly  and  a  more  difficult  item  if  he  answers  accurately.  The 
battery  includes  several  variations  of  the  same  test  to  permit  determining  the  effects  of 
such  variables  as  stimulus  intensity  and  forward  and  backward  masking. 


■V* - - - 
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Regression  Analysis  of  Reasons  for  Leaving  the  Navy 


Background 

The  retention  of  qualified  petty  officers  is  a  problem  of  considerable  magnitude  for 
the  Navy.  It  is  important  to  understand  the  reasons  for  separation  from  the  Navy  in  order 
to  provide  recommendations  for  ameliorative  policy  changes.  The  Enlisted  Separation 
Questionnaire  (ESQ)  is  a  data  source  for  identifying  these  reasons.  It  consists  of  10 
background  questions  and  30  items  listing  reasons  for  leaving  the  Navy  (e.g.,  "working 
hours  too  long").  Individuals  respond  to  each  reason  on  a  5-point  scale  ranging  from  "no 
importance"  to  "extremely  important." 

Previous  work  employed  simple  regression  analysis  to  predict  first-term  reenlistment. 
The  present  effort  applied  hierarchical  multiple  regression  (HMR)  to  the  ESQ  data  to 
determine  the  relationship  between  the  number  of  completed  enlistments  and  the  reasons 
for  separation. 

Objectives 

The  objectives  of  this  effort  were  to  determine  (1)  if  the  hierarchical  multiple 
regression  method  is  useful  in  analysis  of  the  ESQ,  and  (2)  the  relationship  between  the 
number  of  completed  enlistments  and  importance  of  separation  reasons  for  specific 
subclasses  of  individuals  as  well  as  for  the  total  group  leaving  the  Navy. 

Approach 

Approximately  6500  enlisted  personnel  completed  the  ESQ  upon  discharge  from  the 
Navy  during  FY80.  Factor  analysis  of  the  ESQ  reasons  for  leaving  the  Navy  yielded  the 
following  eight  factors: 

1.  Leadership. 

2.  Skill  utilization. 

3.  Quality  of  benefits. 

4.  Loss  of  benefits. 

5.  Regimentation. 

6.  Nonpermanent  home. 

7.  Amount  of  work. 

8.  Pay. 

Biodemographic  variables  from  the  ESQ  were  used  to  classify  the  sample  into  mutually 
exclusive  subgroups  (gender,  education,  pay  grade,  etc.).  The  correlation  between  number 
of  completed  enlistments  and  importance  of  each  ESQ  factor  was  then  determined.  HMR 
was  used  to  establish  the  relationship  between  the  number  of  completed  enlistments  and 
each  ESQ  factor  for  the  mutually  exclusive  subgroups. 


Note.  This  work  was  accomplished  under  FY81  work  unit  ZROOO-0 1-042-04-03.02:  The 
Reenlistment  Decision  Process.  A  more  extensive  discussion  of  this  work  will  be  provided 
in  a  NPRDC  TN  (Doherty  ic  Cowen,  in  press). 
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As  shown  in  Figure  2,  the  number  of  completed  enlistments  correlated  positively  with 
loss  of  benefits  (.21),  and  negatively  with  leadership  (-.14),  skill  utilization  (-.24),  and 
regimentation  (-.16).  These  correlations  were  statistically  significant.  As  the  number  of 
enlistments  increases,  loss  of  benefits  becomes  more  important  as  a  separation  reason, 
and  skill  utilization,  leadership,  and  regimentation  become  less  important.  However,  the 
strength  of  the  relationship  between  importance  of  leadership  and  number  of  completed 
enlistments  was  different  for  pay  grade,  education,  and  rating.  Moreover,  the  strength  of 
the  relationship  between  importance  of  regimentation  and  number  of  completed  enlist¬ 
ments  was  different  for  pay  grade,  marital  status,  fleet,  and  gender. 


NUMBER  OF  COMPLETED  ENLISTMENTS 


Figure  2.  Relationships  between  completed  enlistments  and  importance  of  ESQ  factors. 


Delayed  Feedback  In  Acquisition  And  Retention 
Problem  and  Objective 

The  Navy's  computer-managed  instruction  (CMI)  system,  personalized  system  of 
instruction  (PSI),  and  precision  teaching  are  instructional  systems  that  provide  immediate 
feedback  to  strengthen  student  learning.  However,  immediate  feedback  can  be  expensive 
in  terms  of  both  student  time  and  instructor  time.  Evidence  is  mounting  that  delayed 
feedback  produces  equal  learning  and,  frequently,  superior  retention,  at  least  when 
multiple-choice  or  fill-in  test  items  are  used.  The  objective  of  this  series  of  experiments 
was  to  examine  the  relationship  between  the  timing  of  feedback  and  long-term  retention 
under  classroom  conditions. 

Approach 

Three  experiments  were  conducted  in  courses  taught  according  to  PSI  principles. 
Experiment  I  examined  retention  as  a  function  of  feedback-delay  interval,  using  short- 
answer  essay  tests.  Experiment  II  varied  feedback-delay  interval,  informational  quality  of 
feedback,  and  test  item  type.  Experiment  III  examined  delay  and  item  type. 

Findings 


This  series  of  experiments  showed  no  evidence  of  superiority  for  either  immediate  or 
delayed  feedback.  Feedback  delay  did  not  impair  learning  in  any  of  the  three  experiments 
in  terms  of  acquisition,  retention,  or  study  time,  regardless  of  quality  of  feedback  or  test 
item  type  used.  Subjects  in  the  immediate  feedback  conditions  reported  no  more  initial 
errors  than  did  those  in  the  delayed  feedback  conditions.  Further,  different  types  of 
feedback  (varying  the  amount  of  information)  did  not  produce  differential  levels  of 
retention.  The  repeatable  quizzing  aspect  of  PSI  probably  makes  feedback  a  less  potent 
variable  than  it  is  in  other  types  of  courses,  because  students  have  to  learn  smaller 
quantities  of  material  for  each  test  and  have  many  opportunities  to  learn  from  whatever 
type  of  feedback  is  provided.  The  expense  and  trouble  involved  in  providing  immediate 
feedback  do  not  appear  to  be  warranted  in  these  types  of  learning  situations.  Operators 
of  the  Navy  CMI  system  have  continued  to  apply  procedures  consistent  with  these 
findings. 


Note.  This  work  was  accomplished  under  FY81  work  unit  ZROOO-0 1-042-06-0 1.02: 
Delayed  Feedback  in  Acquisition  and  Retention.  A  more  extensive  discussion  of  this  work 
is  provided  in  NPRDC  TR  83-13  (Lockhart,  Sturges,  Zachai,  Dubois,  &  Groves,  1983)  (AD- 
A 125  802). 
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Metaphors  and  Analogies  in  Training 


Background 

Metaphors  and  analogies  have  been  used  as  teaching  devices  at  least  since  the  time  of 
Plato.  Intuition  and  empirical  studies  suggest  that  metaphors  can  increase  comprehen¬ 
sion,  retention,  transfer,  and  interest.  The  key  criticism  of  using  metaphors  and  analogies 
in  instruction  is  that  they  are  imprecise  and  potentially  misleading.  They  also  can  lead  to 
misunderstanding  when  students  do  not  have  the  necessary  prerequisite  knowledge  about 
the  metaphor  or  analogy  to  apply  it  to  the  new  topic. 

Metaphors,  analogies,  and  their  close  relatives,  similies  and  models,  are  one  way  of 
relating  new  knowledge  to  what  is  already  known  and  of  using  one  subject  to  teach 
another  subject.  The  terms  "metaphor"  and  "analogy"  have  distinct  definitions  but,  as 
instructional  strategies,  are  used  interchangeably.  They  mean  any  type  of  figurative 
language,  or  graphic  or  physical  model,  that  describes  one  thing  in  terms  of  another— that 
is,  any  nonliteral  similarity  comparison  used  for  explanation. 

Although  single-word  methaphors  are  found  in  instructional  material,  the  present 
study  addressed  the  macro-level  metaphor:  one  system  or  domain  described  in  terms  of 
another  system  or  domain.  Examples  of  macro-  or  system-level  metaphors  are  light 
explained  in  terms  of  water  waves,  Campbell's  billiard  ball  model  for  the  kinetic  theory  of 
gases,  and  the  hydraulic  model  of  the  blood  circulation  system. 

Purpose  and  Approach 

Given  the  pros  and  cons  of  using  metaphors  in  instructional  materials,  this  effort 
investigated  the  characteristics  of  good  metaphors,  and  when  and  how  they  should  be  used 
to  capitalize  on  their  positive  attributes  while  avoiding  imprecision  and  misunderstanding. 
The  desired  product  was  a  set  of  guidelines  on  the  appropriate  selection  and  use  of 
metaphors  and  analogies  for  instruction.  To  develop  the  guidelines,  two  approaches  were 
taken.  First,  21  computer  and  31  electricity  texts  were  analyzed  to  determine  how 
analogies  were  used  in  teaching  basic  computer  concepts  and  basic  electricity.  Then,  the 
empirical  and  theoretical  research  on  instructional  metaphors,  consisting  of  more  than  50 
experimental  studies  and  100  theoretical  articles,  was  reviewed. 

Results 


Fifteen  prescriptive  guidelines  were  developed  for  using  metaphors  and  analogies  as 
an  instructional  strategy.  The  guidelines  are  not  algorithms;  rather,  they  assist  in 
answering  three  classic  questions  in  curriculum  development:  what  to  teach,  how  to  teach 
it,  and  when  to  teach  it.  Guidelines  1  through  9  specify  what  the  characteristics  of 
appropriate  analogies  are,  10  through  12  specify  how  analogies  should  be  incorporated  into 
instructional  material,  and  13  through  15  specify  when  analogies  should  be  used.  These 
guidelines  are  listed  below. 

1.  Choose  an  analogy  for  a  superordinate  concept  over  different  analogies  for  each 
of  the  subordinate  concepts. 


Note.  This  work  was  accomplished  under  FY82  work  unit  ZROOO-0 1-042-06-0 1.04:  Meta- 
phors  and  Analogies  in  Training. 
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2.  Choose  an  analogy  that  is  no  more  complex  than  the  subject  matter. 

3.  Choose  an  analogy  from  the  students'  experiences;  that  is,  an  analogy  with  high 
familiarity. 

4.  Choose  an  analogy  that  comes  from  a  distinctly  different  field  of  knowledge  than 
the  subject  matter. 

5.  Choose  an  analogy  with  high  imagery  and  high  concreteness  over  one  with  low 
imagery  and  low  concreteness. 

6.  Avoid  analogies  with  connotations  that  are  counterpurpose. 

7.  When  teaching  a  concept,  use  an  analogy  that  shares  more  relevant  attributes 
and  fewer  irrelevant  attributes  with  that  concept.  When  teaching  a  rule  or  principle,  use 
an  analogy  that  shares  more  of  the  relevant  relationships  (operations)  among  the  concepts 
comprising  the  rule  or  principle. 

8.  If  necessary,  use  an  unfamiliar  analogy  only  if  its  content  is  taught  before  the 
analogy  is  used,  and  if  the  gain  in  understanding  is  more  than  the  added  burden  of 
explanation. 

9.  Use  multiple  analogies  for  important  or  complex  concepts  and  principles.  Use  to 
clarify  relevant  and  irrelevant  attirbutes,  emphasize  and  clarify  selected  relationships, 
and  supply  "missing"  relevant  relationships. 

10.  Use  analogies  as  advance  organizers,  introductions,  and  helps. 

11.  Provide  all  necessary  cues  to  the  inclusion  of  an  analogy.  Reduce  reliance  on 
student's  learning  strategies.  (This  is  a  key  difference  from  the  use  of  metaphors  in 
literature.) 


a.  Identify  the  analogy  as  an  analogy.  Distinguish  between  the  analogy  and  the 
subject  matter  or  examples. 

b.  Specify  that  the  analogy  involves  a  comparison  and  state  the  basis  for  the 
comparison.  Specify  the  limits  of  the  generalization. 

c.  Specify  how  the  analogy  can  mold  one’s  thinking. 

12.  Move  beyond  the  analogy  to  the  formal,  literal  definitions  and  rules. 

13.  Use  an  analogy  when  the  material  cannot  be  readily  assimilated  into  the 
student's  existing  knowledge  structure.  The  analogy  should  establish  a  knowledge 
structure. 

14.  An  analogy  is  most  helpful  for  low-ability,  inexperienced  students. 

15.  An  analogy  is  more  useful  for  conceptual  understanding  than  for  rote  learning.  A 
mnemonic  or  other  memory-aiding  device  is  preferred  for  teaching  a  fact. 
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Oculomotor  Training  For  Pilots 


Objective 

The  objective  of  this  continuing  effort  is  to  evaluate  a  training  program  designed  by 
the  Athletic  Perception  Institute  (API)  to  improve  the  oculomotor  (eye  movement)  skills 
of  Navy  pilots. 

Approach 

Forty  VF-124  student  pilots  were  pretested  on  a  device  that  measured  their  eye 
movement  skills.  They  were  then  assigned  to  one  of  two  groups  matched  for  experience 
and  pretest  scores.  One  group,  the  experimental  group,  received  API’s  oculomotor 
training  program,  which  consists  of  30  training  sessions  on  recognition  of  visual  stimuli 
projected  tachistoscopically  via  two  eye-movement  devices,  Tach  Eye  and  Guided  Eye. 
The  other  group,  the  control  group,  received  no  special  eye-movement  training.  Both 
groups  were  given  a  posttest  on  their  eye-movement  ability.  Student  records  on  all 
subjects  are  being  examined  to  reveal  any  differences  in  ability  during  critical  parts  of 
the  VF-124  training  pipeline  that  might  be  related  to  eye-movement  skills. 
Recommendations  for  further  work  will  then  be  developed  based  on  the  findings. 

Results 


The  results  of  the  training  sessions  on  the  experimental  group  are  shown  in  Figures  3 
and  4.  Both  the  Tach  Eye  and  Guided  Eye  scores  improved  as  a  result  of  training.  No 
significant  correlations  were  found  between  pretest  or  posttest  scores  and  the  number  of 
flight  hours  reported  by  the  trainees.  Average  pretest  and  posttest  scores  for  the  15 
experimental  subjects  who  participated  in  both  the  pretest  and  the  posttest  were  56.58 
and  68.37  respectively,  compared  to  56.16  and  57.09  for  the  8  control  subjects  who 
participated  in  both  tests. 

An  analysis  of  variance  on  the  pretest  and  posttest  scores  showed:  (1)  no  significant 
differences  between  the  two  groups  of  subjects  in  their  pretest  eye  movement  scores,  (2) 
a  significant  increase  in  test  scores  between  the  pretest  and  the  posttest  in  the 
experimental  group,  and  (3)  a  significant  difference  between  posttest  scores  of  the 
experimental  group  and  control  group.  As  a  check  on  the  durability  of  the  effect,  a 
second  posttest  was  administered  to  a  small  sample  of  subjects  2  months  after  training 
had  been  completed.  Although  the  sample  was  too  small  to  permit  statistical  verification, 
the  second  posttest  results  appeared  to  be  identical  to  the  first  results,  suggesting  that  no 
decrease  in  skills  had  occurred. 

Conclusions 


The  oculomotor  training  had  a  significant  effect  on  the  eye-movement  abilities  of  the 
subjects  who  received  the  training.  Conclusions  about  the  utility  of  this  effect  are 
dependent  on  data  for  each  subject  from  the  critical  parts  of  the  VF-124  training  syllabus. 

Remaining  Tasks 

All  students  will  have  completed  training  by  the  end  of  April  1983.  Student  records 
will  be  examined  as  they  become  available.  Recommendations  will  then  be  formulated 
and  COMNAVA1RPAC  will  be  briefed  concerning  any  further  work  warranted  by  the 
findings. 


Note.  This  work  was  accomplished  under  FY82  work  unit  ZR 000-0 1-04 2-06-0 1.06:  Oculo¬ 
motor  Training  for  Pilots. 
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3.  Tach  eye  performance  improvement  with  training. 
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Figure  4.  Guided  eye  performance  improvement  with  training, 


Acquisition  Project  Management 


Problem  and  Purpose 

The  acquisition  of  the  Navy's  weapon  platforms,  systems,  and  equipments  is  a 
complex  and  involved  business.  The  Naval  Material  Command  and  its  systems  commands 
are  responsible  for  managing  the  Navy's  acquisition  process  through  project  management; 
that  is,  each  major  system  acquisition  has  a  project  organization  with  a  project  manager 
and  a  staff  to  manage  it  through  the  acquisition  process. 

The  purpose  of  this  effort  was  to  investigate  the  different  roles  of  the  project 
manager  and  his  staff  with  a  view  toward  identifying  possible  intrarole  conflicts  (e.g.,  a 
task  that  is  incompatible  with  the  other  tasks  in  the  same  role  or  with  other  roles  that  the 
incumbent  has  to  perform).  This  project  was  initiated  to  develop  knowledge  to  support 
future  R&D  related  to  acquisition  project  management. 

Approach 

The  approach  was  to  review  literature,  documents,  and  other  written  materials,  and 
to  interview  project  management  personnel  in  the  various  support  organizations  within  the 
SYSCOMs. 

Two  difficulties  were  encountered  in  this  effort.  First,  there  is  a  lack  of  uniformity 
in  the  definitions  of  key  concepts,  titles,  and  words  used  in  the  acquisition  community 
(e.g.,  a  "project  manager"  in  one  SYSCOM  is  a  "program  manager"  in  another).  Second, 
there  is  a  lack  of  reported  research  on  any  project  personnel  other  them  the  project 
manager.  Contract  administrators,  engineering  managers,  systems  managers,  and  other 
second-echelon  personnel  apparently  have  not  been  the  subject  of  research.  This  effort, 
therefore,  was  limited  primarily  to  the  role  of  the  project  manager  and  the  SYSCOM 
organizations. 

Findings 

Some  of  the  finding?,  were  not  expected.  Of  490  acquisition  projects  identified 
through  documentation,  less  than  half  were  being  managed  within  the  project  management 
offices  (PMOs).  The  majority  of  the  projects  are  in  the  functional  organizations  of  the 
SYSCOMs.  Project  offices  tend  to  be  associated  with  large  unique  acquisitions  such  as  a 
class  of  ship  or  model  of  aircraft,  whereas  the  functional  organizations,  being  more 
technically  oriented,  manage  the  acquisition  of  systems  that  are  common  across  weapon 
platforms  (e.g.,  jet  engines  for  aircraft  or  propulsion  plants  for  ships). 

Although  military  personnel  make  up  less  than  6  percent  of  the  Navy's  acquisition 
community,  58  of  the  61  major  PMOs  have  a  military  project  manager.  It  is  widely 
believed  that  routine  reassignment  of  the  military  breaks  the  continuity  of  the  project 
leadership;  however,  it  was  found  that  the  deputy  project  managers,  who  usually  are 
civilians,  also  exhibit  a  high  turnover  rate.  Leadership  discontinuity  is  not  unique  to  the 
military  in  the  acquisition  community.  Specified  acquisition  career  ladders  for  civilians 


Note.  This  work  was  accomplished  under  FY82  work  unit  ZR000-01-042-08-03.02:  Organ¬ 
izational  Effectiveness  of  Program  Management  Offices.  It  has  transitioned  from  IR  to  a 
reimbursable  effort  for  the  Chief  of  Naval  Material:  Acquisition  Skills  Inventory 
Evaluation. 


do  not  exist,  whereas  the  military  have  selection  and  rotation  programs  for  developing 
acquisition  leadership.  Very  few  of  the  project  top  management  people,  both  military  and 
civilian,  ever  attend  formal  acquisition  training,  such  as  the  22-week  acquisition  manage¬ 
ment  course  at  the  Defense  System  Acquisition  Management  College.  The  two  reasons 
given  are  that  the  course  ties  up  executive  talent  too  long,  and  it  is  only  offered  twice  a 
year. 

The  literature  describes  project  management  as  being  different  from  that  practiced 
in  the  traditional  hierarchical  organization.  Project  management  usually  requires  more 
coordination  of  professional  experts,  such  as  engineers,  contract  administrators,  and 
business  managers,  in  areas  of  greater  uncertainty  than  is  typically  found  in  the 
production-oriented  hierarchical  organizations.  The  project  managers,  therefore,  must 
rely  on  their  ability  to  persuade  and  negotiate  and  to  use  other  professionals  or 
organizations  to  accomplish  project  activities.  The  more  technically  competent  the 
manager,  the  more  successful  he  is  at  project  management. 

Presently,  three  criteria  are  cited  for  judging  project  outcome,  viz.,  cost,  schedule, 
and  system  performance.  Many  of  the  factors  that  influence  these  criteria  are  beyond 
reasonable  control  of  the  project  managers  and  their  staffs.  Before  any  practical 
research  on  the  roles  and  tasks  comprising  acquisition  management  can  be  considered, 
additional  performance  criteria  will  have  to  be  developed. 
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INDEPENDENT  EXPLORATORY  DEVELOPMENT  OED)  PROGRAMS 


The  Independent  Exploratory  Development  (IED)  program  at  NAVPERSRANDCEN 
was  initiated  in  FY76.  In  FYs  81  and  82,  programs  were  funded  in  the  areas  listed  in 
Table  2.  Selected  projects  within  these  areas  are  described  in  the  remainder  of  this 
section. 

1R/IED  work  units  for  FY83  are  presented  in  Appendix  A;  and  1R/1ED  publications  and 
presentations,  in  Appendix  B. 


Table  2 

IED  Work  Units  for  FY81-82 


Independent  Exploratory  Development 

Dollars  (in 
thousands) 
FY81  FY82 

ZF66- 5 12-001: 

IED  SUPPORT  TECHNOLOGY 

-010 

Evaluation  of  Psychobiological  Methods 

L5 

5 

-010-03.02 

Color-specific  Visual  Event-related 
Brain  Potentials  as  Related  to 

Navy  Display  Systems 

Dr.  G.  Lewis 

Code  17 

15 

5 

-050 

Factors  Affecting  the  Acceptance  of  Change 

45 

50 

-050-03.08 

Impediments  to  the  Implementation 
of  Innovation3 

Dr.  J.  Sheposh 

Ms.  V.  Hulton 

Code  16 

45 

50 

-070 

Methods  for  Improving  Maintenance  Procedures 

15 

0 

-070-03.07 

Remote  Expertise  for 

Maintenance 

Mr.  W.  Nugent 
Code  17 

15 

0 

-080 

Production  and  Use  of  Graphics 

50 

80 

-080-03.08 

Theory  of  Graphic  Representation 

Dr.  W.  Montague 
Dr.  3.  Hollan 

Code  13 

35 

40 

-080-03.09 

Producing  Effective  Technical 
Graphics 

Dr.  T.  Curran 

Code  13 

15 

40b 

-090 

Human  Performance 

0 

10 

-090-03.01 

Human  Performance  in  Detecting  a 
Change  in  Target  Location 

Dr.  R.  Kelly 

Code  17 

0 

10 

TOTAL 

125 

145 

aDescribed  in  detail  in  previous  IR/IED  report,  NPRDC  SR  82-27. 
bBeing  continued  under  FY83  work  unit  ZF66-512-00I.004  (see  Appendix  A). 


Visual  Event-related  Brain  Potentials 


Background 

Traditional  personnel  assessment  procedures  have  depended  heavily  on  paper-and- 
pencil  tests.  Although  these  measures  have  predicted  school  and  training  performance 
quite  adequately,  their  effectiveness  in  predicting  job  performance  is  less  than  desirable. 
This  shortcoming  is  undoubtedly  due,  at  least  in  part,  to  the  difficulty  in  defining  job 
performance  and  in  devising  reliable  and  valid  measures  of  criterion  variables.  While 
continuing  attention  must  be  dedicated  to  imparting  greater  clarity  and  specificity  to  the 
meaning  of  job  performance,  it  is  believed  that  concurrent  research  emphasizing  "process" 
rather  than  "content"  information  may  be  able  to  improve  the  prediction  of  job 
performance  and,  in  addition,  the  design  and  use  of  system  equipment  and  use.  To  this 
end,  this  Center's  Biotechnology  Laboratory  was  established  with  the  primary  objective  of 
exploring  the  feasibility  of  developing  and  using  a  new  technology  in  personnel  assess¬ 
ment;  namely,  bioelectric  and  biomagnetic  recordings  from  human  brain,  heart,  and 
muscle.  The  major  emphasis,  to  date,  has  been  the  development  of  methods  and 
procedures  to  improve  the  prediction  of  on-job  performance.  Advances  in  the  under¬ 
standing  of  brain  processes  have  provided  procedures  to  assess  individual  differences  and 
unique  capabilities  more  accurately.  Bioelectric  measures,  such  as  electroencephalo¬ 
grams  (EEGs)  and  brain  event-related  potentials  (ERPs),  are  very  small  signals  recorded 
from  the  scalp.  The  EEG  shows  ongoing  brain  activity,  while  the  ERP  shows  activity  after 
vision  and  hearing  have  been  stimulated  (e.g.,  by  light  flashes  or  clicks  to  the  ears).  Brain 
processing  may  be  assessed  through  ERP  procedures  and  may  provide  more  accurate  and 
predictive  information  about  human  performance  than  methods  currently  being  used  by 
the  Navy. 

Results  of  this  Center's  research  in  laboratory  and  operational  settings  suggest 
relationships  between  ERP  measures  and  various  measures  of  academic  and  job  perfor¬ 
mance.  In  the  academic  realm,  our  work  suggests  relationships  between  ERP  measures 
and  success  in  a  Navy  remedial  reading  program,  Navy  paper-and-pencil  aptitude  test 
scores,  cognitive  information  processing,  and  reading  ability.  In  research  related  to  fleet 
performance,  which  includes  recording  ERP  brain  activity  of  pilots  and  radar  intercept 
officers  (RIOs),  differences  were  found  between  the  ERP  measures  of  pilots  and  RIOs. 
For  both  groups,  ERP  records  were  related  to  job  performance.  Also,  ERP  records  were 
shown  to  be  potentially  useful  in  predicting  the  performance  of  sonar  operators  and  in 
assessing  recruit  promotion  rate  and  attrition  better  than  paper-and-pencil  test  scores.  A 
main  thrust  of  the  research  is  to  determine  the  feasibility  of  using  brain  activity  measures 
to  describe  baseline  performance  and  performance  under  psychological  stress. 

Brain  Activity  and  Visual  Processing 

The  increased  sophistication  and  use  of  video  display  systems  in  Navy  ship  and 
aircraft  control  systems  place  greater  demands  on  the  personnel  who  operate  them. 
Recent  research  has  shown  that  ERP  bioelectric  procedures  may  be  able  to  assess  directly 
the  performance  of  human  visual  and  auditory  systems.  Color  processing,  color-form 


Note.  This  work  was  accomplished  under  FY81-82  work  unit  ZF66-5 12-00 1-010-03.02: 
Color-specific  Visual  Event-related  Brain  Potentials  as  Related  to  Navy  Display  Systems. 
It  has  transitioned  from  1ED  to  reimbursable  support  from  the  Defense  Nuclear  Agency. 


19 


interaction,  contour,  element  density,  visual  field  stimulation,  image  quality,  eye- 
movement  efficiency,  and  information  processing  of  visual  images  may  be  more  readily 
and  accurately  assessed  by  biotechnology  procedures  than  by  verbal  reporting  from  test 
subjects.  The  design  of  presentation  formats  for  display  information  may  be  greatly 
improved  when  sensory  and  information  processing  capabilities  are  considered. 

Recent  major  findings  in  ERP  research  relating  to  t!'e  effects  of  color  and  color 
interaction  on  brain  function  and  perception  have  disclosed  ERP  components  for  three 
basic  color  processes;  presumably,  red,  green,  and  blue.  It  appears  that  the  red  process 
develops  most  rapidly,  followed  by  the  green  and  then  the  blue.  Various  studies  suggest 
that  the  components  may  be  related  to  color  processing  at  different  levels  of  the  visual 
system.  Visual  ERP  data  from  the  Center’s  Biotechnology  Laboratory  have  shown  marked 
differences  among  individuals.  Greater  individual  differences  were  obtained  with  red, 
green,  or  blue  stimuli  than  with  white  stimuli.  The  results  suggest  that  subtle  perceptual 
characteristics  may  be  assessed  through  the  ERP  procedure. 

It  has  been  shown  that  pattern  elements  in  visual  stimuli  produce  specific  response 
components  in  ERP  records  that  vary  with  the  nature  of  the  elements  (e.g.,  linear, 
curved),  density  of  the  elements  (e.g.,  spatial  frequency),  and  the  region  of  the  visual  field 
being  stimulated  (e.g.,  upper  versus  lower  fields).  Sharpness  of  focus,  or  quality  of  image 
on  the  retina,  may  be  assessed  through  the  ERP.  Individual  differences  in  the  ERP  to 
optimal  spatial  frequencies  and  binocular  summation  may  be  important  for  assessing 
personnel  performance  and  man-machine  interface  relationships. 

The  effects  of  color  and  color  interaction  on  brain  function  and  perception  have  been 
demonstrated  in  the  Center's  Biotechnology  Laboratory.  Figure  5  presents  sample  data 
from  two  subjects  showing  the  sensitivity  of  ERP  data  to  individual  differences.  The 
visual  stimulus  was  a  green-black  checkerboard  pattern.  The  eight  channels  of  visual  ERP 
data  for  subject  "A"  are  overlaid  on  those  for  subject  "B."  Calibration,  polarity,  and  time 
base  information  are  displayed  along  with  the  pre-stimulus  waveforms.  Post-stimulus 
waveform  amplitude  (standard  deviation,  SDy  V)  and  mean  (MNpV)  values  were  computed 
and  displayed  for  each  subject's  electrode  site.  The  waveforms  in  the  left  column  were 
derived  from  the  left  hemisphere  (LH).  The  waveforms  from  top  to  bottom  were  from  the 
front  to  the  back  of  the  head  at  frontal  (secondary  association  area— F3),  temporal 
(auditory  reception  area— T 3),  parietal  (primary  association  area— P3),  and  occipital 
(visual  reception  area— Ol)  sites.  Right  hemisphere  (RH)  ERP  data  are  represented  in  the 
right  column  similar  to  LH  data  (F4,  T4,  P4,  02).  The  top  two  values  for  each  site 
represent  the  standard  deviation  and  mean  for  subject  "A,"  and  the  bottom  two  values, 
those  for  subject  "B."  Few  similarities  are  apparent  in  the  ERP  records  of  the  two 
subjects. 

Figure  6  shows  a  single  subject's  ERP  waveforms  produced  by  red  checkerboard 
stimuli  in  the  same  display  format  as  in  Figure  5.  The  data  were  obtained  about  2  hours 
apart.  A  great  amount  of  intrasubject  reliability  may  be  seen,  particularly  at  sites 
located  at  the  back  of  the  head  (parietal/occipital).  Color  stimulation  produces  high 
intrasubject  reliable  waveform  records,  in  contrast  to  high  intersubject  variability. 

Analyses  of  these  records  may  include  a  "template"  or  pattern  recognition  as  well  as 
cross-correlation  and  auto-correlation  procedures.  A  dramatic  way  to  assess  the 
similarities  and  differences  in  ERP  waveforms  has  been  recently  developed  in  our 
laboratory.  Data  are  displayed  and  analyzed  over  the  color  video  monitor.  Either  black 
and  white  hardcopy  or  color  35mm  slides  may  be  made  of  the  displayed  data.  The  color 
videographic  system  can  display  up  to  256  colors  simultaneously.  We  take  advantage  of 
this  capability  by  overlaying  two  separate  displays  of  data  at  one  time  in  different  colors. 
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Figure  5.  Visual  (green  stimuli)  ERP  waveforms  for  two  subjects. 


Figure  6.  Visual  (red  stimuli)  ERP  waveforms  recorded  about  2 
hours  apart  for  a  single  subject. 
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Where  the  waveforms  do  not  overlap,  the  original  color  is  seen  (e.g.,  yellow  and  green  or 
red  and  blue).  Where  the  waveforms  do  overlap,  however,  the  new  combined  color  may  be 
seen,  such  as  purple.  Such  color-coding  makes  it  very  easy  to  see  those  ERP  waveform 
areas  that  are  stimulus-dependent  (similarities)  and  those  that  are  unique  between 
subjects  (differences).  Similarities  and/or  differences  within  a  subject  may  also  be 
observed  when  different  stimuli  are  presented  (e.g.,  visual,  auditory).  Detailed  analyses 
of  ERP  waveform  features  are  underway. 

As  may  be  deduced  from  the  above  account,  most  of  the  Center's  effort  to  date  has 
been  to  develop  a  laboratory  capability  for  recording  and  analyzing  electrical  signals 
indicative  of  brain  and  muscle  activity  and  to  continue  building  a  data  base  for  these 
phenomena  under  a  variety  of  baseline  and  duress  conditions.  It  is  believed  that  results 
already  obtained  linking  these  data  to  measures  of  ability  warrant  continuation  of  this 
effort. 
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Remote  Expertise  for  Maintenance 


Background 

Shore-based  technical  assistance  is  generally  requested  when  a  ship  has  determined 
that  its  own  unit  personnel  are  unable  to  diagnose  and  repair  an  equipment  casualty. 
Although  such  casualties  may  constitute  only  a  limited  portion  of  the  maintenance 
problems  encountered  in  the  fleet,  they  are  especially  detrimental  to  overall  operational 
readiness  because  of  the  lengthy  delays  usually  encountered  before  outside  technical 
assistance  is  provided.  One  possible  way  to  reduce  the  amount  of  time  that  a  ship  spends 
in  a  degraded  state  of  equipment  readiness  at  sea  is  to  provide  a  reliable  and  effective 
two-way  communication  link  between  expert  technicians  on  shore  and  deployed  units  of 
the  fleet  using  modern  communication  satellite  technology. 

The  technological  feasibility  of  establishing  a  remote  communications  assistance 
network  was  demonstrated  in  a  project  sponsored  by  the  Naval  Sea  Systems  Command 
(NAVSEA  62C),  entitled  Shipboard  Engineering  Assistance  for  System  Test  and  Repair  via 
Satellite  (SEASTARS).  The  SEASTARS  project  team  was  successful  in  demonstrating  the 
technological  compatibility  of  various  communications  accessory  equipments  that  would 
support  such  a  network,  as  well  as  the  methodology  to  be  used  in  integrating  these  units 
with  a  commercially  available  communications  satellite  system.  However,  the  project 
team  did  not  demonstrate  (either  directly  or  through  simulation)  the  problem-solving 
capabilities  of  this  performance-aiding  concept.  This  study  was  undertaken  to  provide 
such  a  demonstration  and  to  identify  factors  that  might  affect  the  use  of  such  a  procedure 
when  troubleshooting  Navy  equipment  casualties. 

Approach 

Equipment  casualties  representative  of  those  requiring  outside  technical  assistance 
were  inserted  in  the  AN/SQR-17  sonar  detecting-ranging  set  in  two  demonstration 
sessions.  For  each  session,  an  SQR-17  maintenance  instructor  from  the  Fleet  Anti- 
Submarine  Warfare  Training  Center,  Pacific,  San  Diego  served  as  the  remote  maintenance 
technician  (or  expert),  while  2nd  and  3rd  class  surface  sonar  technicians  (STGs)  from  the 
fleet  served  as  shipboard  (or  attendant)  technicians.  A  two-way  voice  communication  link 
was  established  between  the  expert  and  attendant  technicians  via  an  audio  intercom.  For 
session  1,  a  video  link  was  also  provided  that  allowed  the  expert  to  view  those  portions  of 
the  equipment  that  would  provide  information  of  potential  relevance  to  the 
troubleshooting  situation.  For  session  2,  the  expert  was  instructed  to  use  the  video  link 
only  in  those  cases  where  he  could  no  longer  proceed  with  troubleshooting  based  on  audio 
information  alone. 

The  attendant  technicians  were  stationed  at  the  equipment  site  and  were  responsible 
for  reporting  the  symptoms  associated  with  each  casualty  to  the  expert  technician.  They 
were  also  responsible  for  performing  various  fault  isolation  tests  using  standard  Navy  test 
equipment  and  for  removing  and  replacing  printed  circuit  cards  as  directed  by  the  expert. 

The  expert  technician,  who  was  stationed  in  an  adjacent  room,  was  instructed  to 
advise  the  attendants  verbally  in  troubleshooting  and  repairing  equipment  casualties  and, 


Note.  This  work  was  accomplished  under  FY81  work  unit  ZF66-512-001-070-03.07: 
Remote  Expertise  for  Maintenance.  A  more  extensive  discussion  is  presented  in  NPRDC 
SR  82-13  (Nugent,  1982)  (AD-A111  525).  Further  evaluation  of  this  concept  has  been 
transitioned  from  IED  to  project  element  62763N,  work  unit  522-801-014-03.02. 
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as  applicable,  to  direct  the  television  camera  to  those  portions  of  the  sonar  system  or 
supporting  test  equipment. 


Three  troubleshooting  problems  were  inserted  in  the  SQR-17  for  each  session.  The 
faults  were  identical  in  both  cases  but  their  order  of  presentation  varied  between  sessions. 
For  each  problem,  conversations  between  the  expert  and  attendant  technicians,  as  well  as 
the  total  time  spent  in  troubleshooting,  were  recorded. 

Findings 

The  remote  maintenance  technicains  were  successful  in  solving  five  of  the  six 
problems  inserted  in  the  SQR-17  sonar.  The  problem  that  was  not  solved  was  terminated 
due  to  time  limitations  imposed  by  the  testing  schedule.  Conversation  time  between  the 
expert  and  attendant  technicians  accounted  for  an  average  of  only  17  percent  of  the  total 
time  spent  in  troubleshooting  each  problem.  When  visual  information  was  provided,  the 
expert  technicians  were  interested  in  viewing  only  limited  portions  of  the  sonar  set  and 
supporting  test  equipment.  During  session  2,  an  imprecise  verbal  description  of  the 
equipment  casualty  symptoms  associated  with  one  troubleshooting  problem  resulted  in  the 
expert  technician  pursuing  a  lengthy  and  ineffective  troubleshooting  strategy.  When  the 
expert  requested  video  information  to  confirm  the  casualty  symptoms,  he  was  able  to 
locate  and  solve  the  problem  in  less  than  30  minutes. 

Conclusions 


To  the  extent  that  the  current  study  was  able  to  simulate  the  conditions  under  which 
remote  communications  assistance  would  be  provided,  the  feasibility  of  this  concept  was 
successfully  demonstrated.  The  large  proportion  of  time  spent  in  nonconversational 
activity  for  each  troubleshooting  problem  indicates  that  the  communications  link  could  be 
used  intermittently  during  the  course  of  a  remote  communications  assistance  event. 
Expert  maintenance  technicians  must  be  provided  with  a  precise  verbal  description  of  all 
relevant  symptoms  of  an  equipment  casualty  to  facilitate  timely  and  effective  trouble¬ 
shooting  strategies  when  only  audio  link  remote  communications  assistance  is  employed. 


Graphic  Representation  for  Learning 


Introduction 


It  is  generally  believed  that  graphic  presentations  aid  learning  and  comprehension  and 
create  interest.  Dynamic  computer-controlled  graphical  displays  now  make  it  possible  to 
simulate  not  only  events  but  also  aspects  of  those  events  that  have  important 
instructional  significance.  However,  the  conditions  under  which  the  use  of  graphics  will 
enhance  instruction  have  yet  to  be  described  theoretically  and  substantiated  empirically. 

Like  any  notational  system,  graphics  are  better  at  representing  and  communicating 
certain  types  of  information  than  are  other  types.  Thus,  the  effectiveness  of  graphics 
depends  upon  the  kind  of  information  to  be  conveyed,  as  well  as  the  capability  of  the 
person  to  whom  that  information  is  presented  to  understand  the  representation. 

One  primary  reason  for  not  designing  graphics  into  instruction  is  the  lack  of  an 
integrated  cognitive  theory  of  graphical  presentation.  In  addition,  the  use  of  graphics  can 
add  substantially  to  the  cost  of  instruction,  especially  if  the  presentation  is  dynamic  and 
interactive.  However,  the  cost  of  including  dynamic  graphical  presentations  with  which 
the  student  can  interact  is  declining  rapidly  as  the  cost  of  computer  capability  declines. 

Problem 


Recent  research  reviews  indicate  that  the  inclusion  of  graphical  instructional 
material  does  not  in  itself  guarantee  an  increase  in  instructional  effectiveness. 
Reviewers  point  out  that  the  increase  in  instructional  effectiveness  is  largely  assumed. 
The  topics  usually  addressed  are  how  to  select  the  "best"  graphical  medium  and  how  to 
design  displays  for  presenting  graphic  material.  The  reviewers  also  conclude  that  there 
are  conditions  under  which  graphics  may  be  an  important  adjunct  to  the  instructional 
process,  but  currently  there  is  no  way  to  identify  these  conditions. 

Graphics  and  other  media  cannot  be  studied  profitably  by  themselves  as  has  been 
presupposed  in  the  research  literature.  Any  approach  must  conceptualize,  classify,  and 
analyze  what  is  to  be  learned  or  communicated,  determine  which  media  are  appropriate, 
and  relate  these  factors  to  the  cognitive  processes  of  the  learners. 

The  essential  differences  between  different  media  or  forms  of  presentation  are  not 
the  physical  differences  in  the  ways  signals  are  presented,  such  as  television  or  films  or 
static  pictures.  Rather,  the  differences  are  in  the  ways  in  which  particular  presentations 
structure,  convey  contents,  and  influence  the  cognitive  structures  in  a  learner's  mind. 
Various  media  may  convey  the  same  symbols  and,  therefore,  the  same  content.  There  has 
been  little  research  on  how  symbols  affect  mental  representations  and  how  they  function 
to  develop  mental  skills.  Furthermore,  there  has  been  little  investigation  of  the 
interaction  between  a  student's  already  developed  cognitive  skills  and  knowledge  and  the 
effectiveness  of  various  forms  of  symbolic  presentation  on  new  learning  and  performance. 

An  important  idea  drawn  from  work  in  artificial  intelligence  is  that  a  key  step  is  to 
represent  a  problem  in  a  way  that  makes  the  solution  transparent.  The  attempt  is  to  map 
the  task  onto  the  strategies  available  for  problem  solution.  A  format  must  be  found  that 


Note.  This  work  was  accomplished  under  FY81-82  work  unit  ZF66-5 12-001-080-03.08: 
Theory  of  Graphic  Representation.  Evaluation  of  the  programs  developed  to  provide 
instruction  in  maneuvering  board  problem  solving  has  transitioned  from  IED  to  program 
element  62763N,  work  unit  RF63-522-801-013-03.03:  Computer-based  Techniques  for 
Training  Procedural  Skills. 
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permits  a  person  to  perform  certain  tasks  that  would  be  difficult  or  impossible  if 
presented  in  another  format.  For  example,  mathematical  and  geometrical  diagrams  are 
meant  to  aid  problem  solving,  as  are  scientific  diagrams,  tables,  nomograms,  and  the  like. 
Events  or  processes,  normally  invisible  or  unseeable,  help  learners  understand  them  (and 
perhaps  represent  them  cognitively)  and  the  consequences  of  various  actions.  What  is 
needed  is  a  design  technology  to  guide  the  specification  of  graphic  formats  that  will  aid 
people's  understanding  and  problem  solutions. 

Objectives 


The  objectives  of  this  project  were  to  review  the  literature  and  summarize  the  state 
of  our  understanding  regarding  learner  representation  of  knowledge,  to  suggest  how 
graphical  presentation  can  be  used  to  communicate  the  representations  needed  to  perform 
tasks,  and  to  conduct  empirical  demonstrations  of  the  effectiveness  of  new  forms  of 
representing  material  for  learning. 

Approach  and  Results 

We  undertook  to  develop  and  test  the  effectiveness  of  a  different  analogical 
representation  of  simple  electronic  circuits  encountered  by  Navy  recruits  in  the  Basic 
Electricity  and  Electronics  (BE/E)  course.  In  addition,  we  developed  an  interactive 
program  showing  relative  motion  on  a  radar-like  display  along  with  the  same  information 
displayed  on  a  geographic  plot.  This  was  intended  to  facilitate  student  understanding  of  a 
relative  motion  required  of  personnel  such  as  tactical  action  officers  and  operation 
specialists. 

Graphical  Representations  of  Relative  Motion.  A  microcomputer-based  interactive 
graphics  program  was  constructed  to  provide  conceptual  instruction  in  maneuvering  board 
problem  solving.  The  maneuvering  board  is  a  job  aid  to  help  determine  how  to  maneuver 
ships  in  close  proximity  to  one  another.  In  current  formal  training  on  maneuvering  board 
use  and  the  solution  of  relative  motion  problems,  most  failure  rates  are  about  20-30 
percent  for  both  officers  and  enlisted  personnel.  The  difficulty  appears  to  be 
understanding  the  basic  principles  of  relative  motion.  Prior  research  suggested  that 
students  trained  with  a  conceptual  model  that  demonstrated  the  relation  between  relative 
and  geographic  motion  had  better  recall  for  the  procedures. 

The  initial  development  was  a  computer-based  system  that  provided  students  with  a 
conceptual  model  of  the  relationships  between  events  in  the  real  geographic  world  and 
elements  of  the  maneuvering  board  plot.  A  simulation  program  provided  a  bit-map 
graphic  display  of  relationships  between  relative  motion  and  geographic  plots  of  ship 
movements  on  a  TERAK  microcomputer.  The  graphics  display  design  was  based  on  an 
analysis  of  the  conceptual  difficulties  experienced  by  failing  students. 

The  system  was  used  on  a  pilot  basis  at  the  Fleet  Combat  Training  Center,  Pacific. 
Instructors  found  it  useful  as  a  tutorial  aide  when  working  with  students  having  difficulty 
in  learning  the  material.  In  its  original  form,  the  simulation  was  difficult  for  students  to 
use  (Hutchins  <5c  McCandless,  1982).  This  resulted  from  requirements  for  typing  and 
moving  the  cursor  around  on  the  screen.  The  programs  were  later  transferred  to  a  more 
powerful  microcomputer  system  with  better  graphics  and  a  better  interface  with  students. 
In  addition,  the  TERAK  version  was  further  refined  to  be  easier  for  students  to  use. 


Graphical  Representations  of  Basic  Electronics  Concepts.  It  is  common  in  instruction 
to  use  analogies  to  teach  students  new  material.  A  main  problem  is  that  a  particular 
analogy  can  help  or  can  hinder  learning,  depending  on  several  factors.  For  example,  the 
congruence  between  objects  and  object  attributes  in  the  material  to  be  taught  and  those 
in  the  analogy  determine  much  of  the  analogy's  effectiveness,  in  addition,  the  familiarity 
or  understandability  of  the  relations  or  processes  in  the  analogy  to  the  learner  is 
important.  If  students  don't  understand  the  analogy  itself,  it  is  of  doubtful  value  for 
learning  new  material.  Also,  the  analogies  used  to  represent  novel  materials  to  a  new 
learner  may  need  to  be  quite  different  than  those  used  for  a  more  experienced  or 
knowledgeable  person. 

Under  contract  with  NAVPERSRANDCEN,  researchers  at  the  Learning  Research  and 
Development  Center  (LRDC)  of  the  University  of  Pittsburgh  studied  some  aspects  of  this 
problem  using  materials  taken  from  the  Navy's  BE/E  course.  Students  traditionally  have 
had  problems  in  learning  some  of  the  basic  concepts  taught  in  this  course.  For  example,  a 
significant  proportion  of  the  attrition  from  the  course  seems  to  be  due  to  failure  to 
understand  how  combination  series-parallel  circuits  work.  Changes  made  to  course 
materials  that  have  not  recognized  this  problem  have  had  little  effect.  The  LRDC  work 
found  that  students  don't  understand  the  analogies  used  to  teach  about  circuit  relations, 
make  wrong  inferences  about  events  occurring  because  of  changes  in  the  circuit,  and 
make  consistent  errors.  This  led  to  the  idea  that  a  different  representation  or  analogy 
might  be  more  appropriate  for  these  learners. 

The  BE/E  course  uses  a  version  of  the  "moving  crowd"  analogy  to  teach  the  relations 
among  current,  voltage,  and  resistance  in  simple  series  circuits.  Students  are  told  to 
think  of  current  as  the  rate  at  which  a  crowd  of  objects  (electrons,  balls)  moves  along  a 
path.  The  voltage  applied  from  the  battery  corresponds  to  the  push  or  force  to  get  the 
objects  in  motion.  Resistance  is  to  be  thought  of  as  friction  and  depicted  by  a  rough  or 
crooked  path.  The  moving-crowd  analogy  leads  to  some  straightforward  predictions  about 
the  relations  between  voltage,  resistance,  and  current:  (1)  as  voltage  increases  or 
decreases,  current  increases  or  decreases,  (2)  as  resistance  increases  or  decreases,  current 
decreases  or  increases.  The  analogy  makes  no  clear  predictions  about  circuit  relations  at 
the  level  of  individual  resistors,  however,  and  even  leads  to  some  incorrect  predictions. 
For  example,  when  current  flows  through  a  resistor,  there  is  a  drop  in  voltage  across  that 
resistor  directly  proportional  to  the  rate  of  current  flow  and  the  amount  of  resistance. 
The  sum  of  these  voltage  drops  across  all  resistors  in  series  equals  the  voltage  applied. 
Furthermore,  current  is  the  same  at  all  points  in  a  series  circuit.  This  is  not  clear  in  the 
moving-crowd  analogy.  In  fact,  the  opposite  might  be  predicted  if  friction  (resistance)  is 
thought  to  impede  or  slow  current  flow.  Thus,  current  would  be  expected  to  slow  down  as 
it  passes  each  resistor.  Similarly,  since  voltage  is  "dropped"  at  each  resistor,  and  voltage 
is  what  "pushes"  the  crowd  (current),  one  might  expect  current  to  slow  as  it  moves 
through  the  circuit.  Both  of  these  inferences,  one  based  on  friction  and  the  other  on 
voltage  loss,  violate  the  constraint  that  current  is  the  same  everywhere  in  a  series  circuit. 
Errors  like  these  are  common  among  students  learning  BE/E. 

A  new  graphical  representation  of  series  circuits  developed  at  LRDC  is  easily 
understood  by  students  because  of  its  familiarity  and  concreteness.  It  graphically  depicts 
the  simultaneous  constraints  at  both  the  entire  circuit  level  and  at  the  local  resistors. 
Voltage  is  represented  as  a  stack  of  (poker)  chips;  and  resistors,  as  boxes  containing  slots 
equal  to  the  resistance.  In  a  series  circuit,  the  chips  are  distributed  equally  among  the 
resistor  slots.  As  illustrated  in  Figure  7,  R1  is  2  ohms  (a  box  with  two  slots)  and  R2  is  3 
ohms,  for  a  total  of  5  ohms  in  the  circuit.  When  10  volts  (chips)  are  applied,  they  are 
distributed  equally  among  the  slots.  The  2  chips  per  slot  represents  2  amperes  of  current 
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in  the  circuit.  Voltage  drop  for  a  resistor  is  shown  by  the  total  number  of  chips  in  each 
resistor's  slots  (i.e.,  4  volts  for  R1  and  6  volts  for  R2).  This  model  maps  geographically 
into  the  constraints  specified  in  Ohm's  Law  equations. 


Figure  7.  Graphical  representation  of  series  circuits. 


NAVPERSRANDCEN  researchers  adopted  this  approach,  redesigned  materials 
selected  from  the  first  part  of  the  BE/E  course,  and  arranged  them  into  two  sets  of 
workbooks.  One  set  maintained  the  standard  analogies  and  circuit  representation.  In  the 
other,  the  chips  representation  was  added  as  a  learning  aid.  Groups  of  students  studied 
the  materials  and  took  the  regular  progress  tests  for  2  days  prior  to  the  time  they  began 
the  BE/E  course. 

In  terms  of  overall  performance  in  the  course,  there  were  no  differences  between  the 
groups.  However,  on  the  few  problems  included  in  the  progress  tests  that  were  diagnostic 
of  the  misinferences  discussed  above,  85  percent  of  the  chips  group  got  such  problems 
correct,  compared  to  50  percent  of  the  standard  group.  Three  important  points  stem  from 
this  work.  First,  systematic  analysis  of  the  relations  between  analogous  representations 
and  the  systems  that  students  are  to  learn  can  reveal  misunderstandings  that  lead  students 
to  make  errors;  these  misunderstandings  make  learning  difficult.  Second,  once  these 
misunderstandings  are  identified,  they  can  be  corrected,  perhaps  by  designing  a  simpler, 
more  concrete  analogy.  Third,  interactive,  diagnostic  testing  procedures  can  assist  in 
probing  student  misunderstandings  and  in  correcting  them.  Diagnostic  testing  is  necessary 
because  errors  of  misunderstanding  are  revealed  only  by  a  few  test  problems.  In  the 
current  course,  such  test  questions  are  asked  infrequently  (10-15%  of  the  items).  It  was 
only  on  the  "diagnostic"  questions  that  they  differed.  Automated  systems  that  can 
present  those  circuit  problems  and  assess  a  student's  reasoning  about  each  part  or 
component  will  considerably  enhance  our  ability  to  detect  learning  problems  and  make 
learning  more  efficient. 

To  this  end,  we  developed  an  interactive  simulation  of  the  model  on  a  micro¬ 
computer  in  the  form  of  a  program  (TESTER)  that  allows  an  instructor  to  develop  tests. 
Various  aspects  of  problem-solving  performance  can  be  recorded.  Both  standard  and  chips 
representation  forms  of  the  test  are  available.  Evaluation  of  this  program  is  continuing. 


Human  Performance  in  Detecting  a  Change  in  Target  Location 


Background 

An  idealized  command  and  control  problem  was  developed  to  investigate  human 
decision-making  performance.  Specifically,  the  task  for  the  observer  was  to  decide 
whether  or  not  a  target  has  changed  location  from  an  earlier  fix.  Briefly,  the  observer 
views  a  sensor  display  that  shows  a  noisy  sample  of  N  reported  locations  of  the  target. 
The  noise  is  produced  by  a  statistical  distribution  (the  circular  normal  density)  and  each 
datum  on  the  display  appears  as  a  dot  that  encodes  the  reported  (x,y)  coordinates.  If  the 
target  has  not  moved,  the  dots  will  tend  to  cluster  around  the  origin,  which  represents  its 
last  known  position.  If  the  target  moved  since  the  last  fix,  the  dots  will  tend  to  cluster 
around  some  point  offset  from  the  origin.  Based  on  the  noisy  sample,  the  observer  must 
decide  whether  or  not  the  target  has  moved  (at  least  R  units)  from  its  last  known  location. 
It  is  assumed  that  the  target  is  stationary  at  the  time  the  data  are  observed  and  that  all 
directions  of  movement  are  equally  likely. 

This  decision-making  task  represents  a  typical  surveillance  problem.  For  example, 
the  "target"  might  be  a  military  force  under  surveillance  or  an  underwater  object  that  is 
subject  to  drift.  We  seek  to  derive  an  optimal  information-processor  for  assessing  this 
decision-making  skill. 

Decision  Theoretic  Analysis 

As  shown  in  Figure  8,  there  are  two  states  of  nature,  SQ  and  SR.  In  state  Sq,  the 

target  is  at  the  origin,  its  original  location.  In  state  SR,  the  target  has  moved  R  units  to 

the  location  0*1*  )  with  direction  of  movement  given  by  the  angle  which  is  a  uniform 
x  y 

random  variable  on  the  interval  [0,  271].  The  problem  is  to  decide  whether  the  target  has 
moved  or  not;  that  is,  whether  Sq  or  SR  is  true. 


R 

Figure  8.  Two  states  of  nature. 

The  task  for  the  decision  maker  is  illustrated  in  Figure  9.  He  has  been  told  that  an 
earlier  fix  placed  the  target  at  the  center  of  the  circle.  At  the  left  is  a  sample  of  later 
data  from  Sq;  and  at  the  right,  a  sample  of  later  data  from  SR.  The  decision  maker  sees 

one  such  display  or  the  other  and  must  decide  whether  Sq  or  SR  is  true;  that  is,  whether 

the  target  has  moved.  Clearly,  in  this  example,  the  display  on  the  right  indicates  that  the 
target  has  moved,  whereas  the  display  on  the  left  indicates  that  it  has  not  moved.  With 
fewer  observations,  this  task  becomes  more  difficult. 


Note.  This  work  was  accomplished  under  FY82  work  unit  ZF66-5 12-00 1-090-03.01: 
Human  Performance  in  Detecting  a  Change  in  Target  Location. 
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Figure  9.  Two  typical  displays  with  N  =  12. 


The  decision  maker  is  required  to  minimize  the  expected  cost  of  decisions,  where  the 
a  priori  probabilities  of  Sq  and  SR  are  p^  and  p^.  Costs  cure  incurred  as  shown  in  the 

following  cost  matrix: 


S 


O 


S 
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Decision: 

"R" 


o 

O 

o 

CR 

o 

Optimal  Decision  Rule 

Standard  Bayesian  analysis  leads  to  the  following  minimum-cost  decision  rule: 

Decide  "R"  (that  the  target  has  moved)  if,  and  only  if,  the  distance  d 
to  the  centroid  of  the  data  exceeds  a  critical  value  d*. 

The  centroid  is  the  point  (x,y)  given  by  the  means  of  the  x  and  y  coordinates.  The  critical 
distance  d*  involves  a  Bessel  function  and  depends  upon  the  sample  size  N,  the  error 
variance  a2,  the  radius  R  of  the  circle,  the  costs  Cq  and  CR,  and  the  priors  pQ  and  pR. 

The  quantity  d*  is  a  decreasing  function  of  N/a2  and,  for  symmetric  costs  and  priors, 
approaches  R/2  in  the  limit,  as  illustrated  in  Figure  10. 

In  the  context  of  the  theory  of  signal  detectability  (Green  &  Swets,  1966),  we  may 
regard  the  movement  of  the  target  as  a  "signal"  to  be  detected  and  obtain  the  probability 
of  detection  and  the  probability  of  a  false  alarm.  These  are,  respectively,  the 
probabilities  that  the  distance  d  to  the  centroid  of  the  data  exceeds  d*,  given  either  S-  or 

So- 

Given  SQ,  the  quantity  Nd2/o2  turns  out  to  be  a  chi-square  random  variable  with 
2df.  Given  SR,  the  same  quantity  is  a  noncentral  chi-square  random  variable  with  2df  and 
noncentrality  parameter  R2 . 
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Figure  10.  Critical  distance  d*  as  a  function  of  R  and  N/o  . 

Figure  11  exhibits  the  receiver  operating  characteristic,  which  plots  in  the  unit 
square  the  detection  probability  versus  the  false  alarm  probability.  The  curves  shown  are 
for  sample  sizes  N  =  1,  3,  5,  7,  9  with  R  =  1.2  units  and  o2  =  1.  For  a  given  sample  size,  a 
decision  maker  who  uses  the  distance  statistic  d  is  constrained  to  the  given  curve.  The 
points  marked  "o"  designate  optimal  performance  for  the  case  with  symmetric  costs  and 
prior  probabilities. 


0  .9  1 

PRO*  (FALSE  ALARM) 


Figure  11.  ROC  curves  for  R  =  1.2,  o2  =  1,  and  sample  sizes  N  =  1, 
3,  5,  7,  and  9. 


AS-1F  Detector 


The  optimal  decision  maker  quite  properly  considers  all  directions  0  of  the  target's 
possible  movement.  A  quite  different  detector  is  now  considered  that  forms  the 

A  —  — 

maximum  likelihood  estimate  0  =  arctan  (Y/X)  and  then  tests  the  origin  against  the  simple 

A 

alternative  that  the  target  has  moved  R  units  at  the  angle  0,  This  decision  maker  behaves 

A 

"as  if"  0  were  the  only  possible  0  and  is  called  an  AS-IF  detector. 


Analyzing  this  detector  for  (Cj^Pq)/(C0p^)  =  1  yields  the  decision  rule:  Decide  "R" 
if  and  only  if  d  >  R/2.  Note  that  this  rule  does  not  depend  on  N  or  a2  . 

Referring  again  to  Figure  11,  the  digits  1,  3,  5,  7,  9  are,  in  fact,  the  operating  points 
for  the  AS-IF  detector.  This  detector  is  always  on  the  same  curve  as  the  optimal  decision 
maker  because  they  both  use  the  same  test  statistic  d.  However,  the  AS-IF  detector  is 
more  disposed  to  assert  that  the  target  has  moved.  He  will  always  have  higher  detection 
and  false  alarm  probabilities  than  is  optimal  to  minimize  expected  cost. 

Relation  to  Signal  Processing 

After  some  searching,  it  was  discovered  that  the  solution  to  the  spatial  task  of 
detecting  a  change  in  target  location  is  not  an  original  one.  The  solution  may  be  "new" 
but  it  is  identical  to  that  for  a  well-known  problem  in  acoustic  and  radar  signal 
processing— that  of  detecting  in  Gaussian  noise  a  sinusoid  signal  known  exactly  except  for 
phase.  There  is  a  strict  isomorphism  between  these  two  problems;  they  are  indeed  the 
same  problem  and  have  the  same  solution.  The  unknown  phase  of  the  sinusoid  is  the 
unknown  0  in  our  problem.  The  amplitude  of  the  envelope  output  by  the  narrow-band 
filter  is  precisely  the  distance  statistic  that  arises  in  our  target  movement  detection  task. 
The  Rayleigh-Rice  probability  density  functions  that  arise  in  the  signal-processing  domain 
have  as  their  counterparts  the  chi-square  and  noncentral  chi-square  probability  density 
functions. 

Human  Performance 


Some  human  performance  data  were  obtained  for  this  task  with  equal  prior 
probabilities  (so  that  there  is  a  50-50  chance  that  the  target  has  moved).  The  observer 
earns  10  points  for  each  correct  decision  and  loses  10  points  for  each  incorrect  one. 
Three  observers  were  tested  for  approximately  600  plays  at  each  of  the  sample  sizes 
N  =  1,  3,  5,  7,  and  9.  A  microcomputer  drove  a  graphics  terminal  that  always  displayed 
the  circle  of  radius  R  and  plotted  the  N  samples  as  points  in  the  plane.  Feedback-right 
or  wrong— and  the  correct  location  of  the  target  were  given  after  each  play. 

Two  characteristics  of  human  performance  are  of  interest.  First,  the  observers  were 
not  strictly  optimal.  Their  performance  only  approximated  the  ideal,  and,  although  they 
did  better  as  the  sample  size  increased,  they  failed  to  extract  all  the  information 
available  in  the  larger  sample-size  conditions.  Second,  the  observers  were  not  behaving 
like  the  AS-IF  detector.  They  were  using  something  like  the  distance  statistic  d  and 
adjusting  their  criterion  as  the  sample  size  increased.  Figure  12  shows  this  clearly.  Here 
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Figure  12.  Estimated  d*  criteria  for  three  observers  (dashed 
curves)  as  function  of  sample  size.  Optimal  and  "AS- 
IF"  performance  are  plotted  for  comparison. 


the  critical  distance  is  shown  as  a  function  of  N.  The  open  circles  show  how  d*  properly 
decreases  for  the  optimal  detector.  The  d*  criteria  were  estimated  for  each  of  the  three 
observers  by  taking  that  distance  d  for  which  they  were  equally  likely  to  respond  "R"  or 
"O."  These  are  plotted  as  the  three  dashed  curves  in  Figure  12.  Note  that  they  are 
"close"  to  the  ideal  d*,  and  it  is  clear  that  the  AS-IF  detector  must  be  rejected  as  a 
possible  model  for  the  observers. 

Sequential  Decision  Problem 

Up  to  now,  the  sample  size  N  has  been  treated  as  fixed.  Next,  consideration  wi'.i  be 
given  to  the  more  interesting  sequential  decision  problem  in  which  the  decision  maker, 
after  each  observation  (sample),  must  decide  either  "O"  (the  target  is  at  its  original 
locus),  "R"  (the  target  has  moved),  or  elect  to  buy  another  sample  at  some  cost.  Correct 
terminal  decisions  are  worth,  say,  10  points.  Incorrect  terminal  decisions  cost  10  points. 
Let  the  price  of  a  sample  be  1  point.  Again,  equal  priors  are  used  and  the  decision  maker 
is  to  play  the  sequential  game  so  as  to  maximize  his  expected  payoff. 

Now,  the  optimal  decision  maker  will  adopt  a  decision  rule  of  the  form  shown  in 
Figure  13.  For  an  observed  distance  d  to  the  centroid  that  exceeds  the  upper  curve,  the 
decision  maker  decides  that  the  target  has  changed  location.  If  d  is  beneath  the  lower 
curve,  the  decision  maker  decides  the  target  has  not  moved.  For  values  of  d  between  the 
bounds,  the  observer  should  defer  and  buy  another  sample.  The  bounds  are,  of  course, 
dynamic  and  vary  as  a  function  of  the  number  of  samples  already  in  hand. 


Figure  13.  Optimal  decision  bounds  as  a  function  of  sample  size  in 
the  sequential  problem. 


The  Wald  (1947)  approximations  were  used  for  the  decision  boundaries  that  are 
usually  given  in  terms  of  the  desired  error  probabilities,  ct  and  3.  The  "quasi-optimal" 
bounds  were  found  by  running  Monte  Carlo  simulations  based  on  the  Wald  boundaries,  and 
then  searching  the  Wald  space  for  the  maximum  expected  payoff.  Figure  14  shows  these 
optimal  bounds  (the  solid  lines)  for  the  first  six  samples  of  the  sequential  game  in  which 
R  =  1.2  and  <j2  =  1.  Notice  incidentally  that,  for  this  game,  the  optimal  decision  maker 
never  decides  that  the  target  has  not  moved  with  less  than  three  samples  in  hand. 
However,  a  single  sample  with  a  large  distance  from  the  origin  (about  2.75  units)  can 
trigger  the  decision  that  the  target  has  changed  location. 


Figure  14.  Optimal  decision  bounds  (solid  curves)  and  estimated 
bounds  for  human  observer  (o's)  for  the  sequential  game 
with  R  r  1.2,  a2  =  1. 


The  circles  plotted  in  Figure  14  are  the  data  of  a  well-practiced  observer  for  500 
plays  of  this  sequential  game.  The  upper  circles  are  the  observer's  indifference  points 
between  the  terminal  decision  "R"  and  the  decision  to  "buy  another  sample."  The  lower 
circles  are  the  indifference  points  between  the  decision  "O"  and  "buy  another  sample." 
The  dashes  above  and  below  each  point  indicate  the  extent  of  variability  in  the  observer's 
bounds.  For  this  observer,  there  is  surprisingly  good  conformity  with  the  optimal  bounds. 
Data  from  other  observers  also  clearly  follow  the  form  of  the  optimal  prescription. 

Conclusions 

The  quantification  and  understanding  of  human  performance  are  essential  to  the 
design  of  effective  surveillance  systems  for  command  control  operations.  Therefore, 
human  abilities  and  limitations  need  to  be  considered  in  order  to  guide  the  design  of 
improved  surveillance  displays  and  software  algorithms.  In  real  systems,  these  abilities 
and  limitations  are  all  too  often  ignored--and  result  in  expensive  hardware  fixes,  software 
modifications,  or  just  plain  degraded  systems  performance.  The  preliminary  data  given 
here,  however,  speak  quite  well  for  the  human  as  a  spatial  processor  and  decision  maker  in 
a  task  that  is  by  no  means  straightforward. 
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